Skip macroblock coding

ABSTRACT

Various techniques and tools for encoding and decoding (e.g., in a video encoder/decoder) binary information (e.g., skipped macroblock information) are described. In some embodiments, the binary information is arranged in a bit plane, and the bit plane is coded at the picture/frame layer. The encoder and decoder process the binary information and, in some embodiments, switch coding modes. For example, the encoder and decoder use normal, row-skip, column-skip, or differential modes, or other and/or additional modes. In some embodiments, the encoder and decoder define a skipped macroblock as a predicted macroblock whose motion is equal to its causally predicted motion and which has zero residual error. In some embodiments, the encoder and decoder use a raw coding mode to allow for low-latency applications.

RELATED APPLICATION INFORMATION

[0001] The present application claims the benefit of U.S. ProvisionalPatent Application Serial No. 60/341,674, entitled “Techniques and Toolsfor Video Encoding and Decoding,” filed Dec. 17, 2001, the disclosure ofwhich is incorporated by reference. The present application also claimsthe benefit of U.S. Provisional Patent Application Serial No.60/377,712, entitled “Skip Macroblock Coding,” filed May 3, 2002, thedisclosure of which is incorporated by reference.

TECHNICAL FIELD

[0002] Techniques and tools for encoding/decoding binary information invideo coding/decoding applications are described. For example, a videoencoder encodes skipped macroblock information.

BACKGROUND

[0003] Digital video consumes large amounts of storage and transmissioncapacity. A typical raw digital video sequence includes 15 or 30 framesper second. Each frame can include tens or hundreds of thousands ofpixels (also called pels). Each pixel represents a tiny element of thepicture. In raw form, a computer commonly represents a pixel with 24bits. Thus, the number of bits per second, or bit rate, of a typical rawdigital video sequence can be 5 million bits/second or more.

[0004] Most computers and computer networks lack the resources toprocess raw digital video. For this reason, engineers use compression(also called coding or encoding) to reduce the bit rate of digitalvideo. Compression can be lossless, in which quality of the video doesnot suffer but decreases in bit rate are limited by the complexity ofthe video. Or, compression can be lossy, in which quality of the videosuffers but decreases in bit rate are more dramatic. Decompressionreverses compression.

[0005] In general, video compression techniques include intraframecompression and interframe compression. Intraframe compressiontechniques compress individual frames, typically called I-frames, or keyframes. Interframe compression techniques compress frames with referenceto preceding and/or following frames, and are called typically calledpredicted frames, P-frames, or B-frames.

[0006] Microsoft Corporation's Windows Media Video, Version 7 [“WMV7”]includes a video encoder and a video decoder. The WMV7 encoder usesintraframe and interframe compression, and the WMV7 decoder usesintraframe and interframe decompression.

[0007] A. Intraframe Compression in WMV7

[0008]FIG. 1 illustrates block-based intraframe compression (100) of ablock (105) of pixels in a key frame in the WMV7 encoder. A block is aset of pixels, for example, an 8×8 arrangement of pixels. The WMV7encoder splits a key video frame into 8×8 blocks of pixels and appliesan 8×8 Discrete Cosine Transform [“DCT”] (110) to individual blocks suchas the block (105). A DCT is a type of frequency transform that convertsthe 8×8 block of pixels (spatial information) into an 8×8 block of DCTcoefficients (115), which are frequency information. The DCT operationitself is lossless or nearly lossless. Compared to the original pixelvalues, however, the DCT coefficients are more efficient for the encoderto compress since most of the significant information is concentrated inlow frequency coefficients (conventionally, the upper left of the block(115)) and many of the high frequency coefficients (conventionally, thelower right of the block (115)) have values of zero or close to zero.

[0009] The encoder then quantizes (120) the DCT coefficients, resultingin an 8×8 block of quantized DCT coefficients (125). For example, theencoder applies a uniform, scalar quantization step size to eachcoefficient, which is analogous to dividing each coefficient by the samevalue and rounding. For example, if a DCT coefficient value is 163 andthe step size is 10, the quantized DCT coefficient value is 16.Quantization is lossy. The reconstructed DCT coefficient value will be160, not 163. Since low frequency DCT coefficients tend to have highervalues, quantization results in loss of precision but not complete lossof the information for the coefficients. On the other hand, since highfrequency DCT coefficients tend to have values of zero or close to zero,quantization of the high frequency coefficients typically results incontiguous regions of zero values. In addition, in some cases highfrequency DCT coefficients are quantized more coarsely than lowfrequency DCT coefficients, resulting in greater loss ofprecision/information for the high frequency DCT coefficients.

[0010] The encoder then prepares the 8×8 block of quantized DCTcoefficients (125) for entropy encoding, which is a form of losslesscompression. The exact type of entropy encoding can vary depending onwhether a coefficient is a DC coefficient (lowest frequency), an ACcoefficient (other frequencies) in the top row or left column, oranother AC coefficient.

[0011] The encoder encodes the DC coefficient (126) as a differentialfrom the DC coefficient (136) of a neighboring 8×8 block, which is apreviously encoded neighbor (e.g., top or left) of the block beingencoded. (FIG. 1 shows a neighbor block (135) that is situated to theleft of the block being encoded in the frame.) The encoder entropyencodes (140) the differential.

[0012] The entropy encoder can encode the left column or top row of ACcoefficients as a differential from a corresponding column or row of theneighboring 8×8 block. FIG. 1 shows the left column (127) of ACcoefficients encoded as a differential (147) from the left column (137)of the neighboring (to the left) block (135). The differential codingincreases the chance that the differential coefficients have zerovalues. The remaining AC coefficients are from the block (125) ofquantized DCT coefficients.

[0013] The encoder scans (150) the 8×8 block (145) of predicted,quantized AC DCT coefficients into a one-dimensional array (155) andthen entropy encodes the scanned AC coefficients using a variation ofrun length coding (160). The encoder selects an entropy code from one ormore run/level/last tables (165) and outputs the entropy code.

[0014] A key frame contributes much more to bit rate than a predictedframe. In low or mid-bit rate applications, key frames are oftencritical bottlenecks for performance, so efficient compression of keyframes is critical.

[0015]FIG. 2 illustrates a disadvantage of intraframe compression suchas shown in FIG. 1. In particular, exploitation of redundancy betweenblocks of the key frame is limited to prediction of a subset offrequency coefficients (e.g., the DC coefficient and the left column (ortop row) of AC coefficients) from the left (220) or top (230)neighboring block of a block (210). The DC coefficient represents theaverage of the block, the left column of AC coefficients represents theaverages of the rows of a block, and the top row represents the averagesof the columns. In effect, prediction of DC and AC coefficients as inWMV7 limits extrapolation to the row-wise (or column-wise) averagesignals of the left (or top) neighboring block. For a particular row(221) in the left block (220), the AC coefficients in the left DCTcoefficient column for the left block (220) are used to predict theentire corresponding row (211) of the block (210).

[0016] B. Interframe Compression in WMV7

[0017] Interframe compression in the WMV7 encoder uses block-basedmotion compensated prediction coding followed by transform coding of theresidual error. FIGS. 3 and 4 illustrate the block-based interframecompression for a predicted frame in the WMV7 encoder. In particular,FIG. 3 illustrates motion estimation for a predicted frame (310) andFIG. 4 illustrates compression of a prediction residual for amotion-estimated block of a predicted frame.

[0018] The WMV7 encoder splits a predicted frame into 8×8 blocks ofpixels. Groups of 4 8×8 blocks form macroblocks. For each macroblock, amotion estimation process is performed. The motion estimationapproximates the motion of the macroblock of pixels relative to areference frame, for example, a previously coded, preceding frame. InFIG. 3, the WMV7 encoder computes a motion vector for a macroblock (315)in the predicted frame (310). To compute the motion vector, the encodersearches in a search area (335) of a reference frame (330). Within thesearch area (335), the encoder compares the macroblock (315) from thepredicted frame (310) to various candidate macroblocks in order to finda candidate macroblock that is a good match. The encoder can checkcandidate macroblocks every pixel or every ½ pixel in the search area(335), depending on the desired motion estimation resolution for theencoder. Other video encoders check at other increments, for example,every ½ pixel. For a candidate macroblock, the encoder checks thedifference between the macroblock (315) of the predicted frame (310) andthe candidate macroblock and the cost of encoding the motion vector forthat macroblock. After the encoder finds a good matching macroblock, theblock matching process ends. The encoder outputs the motion vector(entropy coded) for the matching macroblock so the decoder can find thematching macroblock during decoding. When decoding the predicted frame(310), a decoder uses the motion vector to compute a predictionmacroblock for the macroblock (315) using information from the referenceframe (330). The prediction for the macroblock (315) is rarely perfect,so the encoder usually encodes 8×8 blocks of pixel differences (alsocalled the error or residual blocks) between the prediction macroblockand the macroblock (315) itself.

[0019]FIG. 4 illustrates the computation and encoding of an error block(435) for a motion-estimated block in the WMV7 encoder. The error block(435) is the difference between the predicted block (415) and theoriginal current block (425). The encoder applies a DCT (440) to errorblock (435), resulting in 8×8 block (445) of coefficients. Even morethan was the case with DCT coefficients for pixel values, thesignificant information for the error block (435) is concentrated in lowfrequency coefficients (conventionally, the upper left of the block(445)) and many of the high frequency coefficients have values of zeroor close to zero (conventionally, the lower right of the block (445)).

[0020] The encoder then quantizes (450) the DCT coefficients, resultingin an 8×8 block of quantized DCT coefficients (455). The quantizationstep size is adjustable. Again, since low frequency DCT coefficientstend to have higher values, quantization results in loss of precision,but not complete loss of the information for the coefficients. On theother hand, since high frequency DCT coefficients tend to have values ofzero or close to zero, quantization of the high frequency coefficientsresults in contiguous regions of zero values. In addition, in some caseshigh frequency DCT coefficients are quantized more coarsely than lowfrequency DCT coefficients, resulting in greater loss ofprecision/information for the high frequency DCT coefficients.

[0021] The encoder then prepares the 8×8 block (455) of quantized DCTcoefficients for entropy encoding. The encoder scans (460) the 8×8 block(455) into a one dimensional array (465) with 64 elements, such thatcoefficients are generally ordered from lowest frequency to highestfrequency, which typical creates long runs of zero values.

[0022] The encoder entropy encodes the scanned coefficients using avariation of run length coding (470). The encoder selects an entropycode from one or more run/level/last tables (475) and outputs theentropy code.

[0023] When the motion vector for a macroblock is zero (i.e., no motion)and no residual block information is transmitted for the macroblock, theencoder uses a 1-bit skip macroblock flag for the macroblock. For manykinds of video content (e.g., low motion and/or low bitrate video), thisreduces bitrate by avoiding the transmission of motion vector andresidual block information. The encoder puts the skip macroblock flagfor a macroblock at the macroblock layer in the output bitstream, alongwith other information for the macroblock.

[0024]FIG. 5 shows the decoding process (500) for an inter-coded block.Due to the quantization of the DCT coefficients, the reconstructed block(575) is not identical to the corresponding original block. Thecompression is lossy.

[0025] In summary of FIG. 5, a decoder decodes (510, 520) entropy-codedinformation representing a prediction residual using variable lengthdecoding and one or more run/level/last tables (515). The decoderinverse scans (530) a one-dimensional array (525) storing theentropy-decoded information into a two-dimensional block (535). Thedecoder inverse quantizes and inverse discrete cosine transforms(together, 540) the data, resulting in a reconstructed error block(545). In a separate path, the decoder computes a predicted block (565)using motion vector information (555) for displacement from a referenceframe. The decoder combines (570) the predicted block (555) with thereconstructed error block (545) to form the reconstructed block (575).

[0026] When the decoder receives a skip macroblock flag for amacroblock, the decoder skips computing a prediction and decodingresidual block information for the macroblock. Instead, the decoder usescorresponding pixel data from the location of the macroblock in thereference frame.

[0027] The amount of change between the original and reconstructed frameis termed the distortion and the number of bits required to code theframe is termed the rate. The amount of distortion is roughly inverselyproportional to the rate. In other words, coding a frame with fewer bits(greater compression) will result in greater distortion and vice versa.One of the goals of a video compression scheme is to try to improve therate-distortion—in other words to try to achieve the same distortionusing fewer bits (or the same bits and lower distortion).

[0028] Although the use of skip macroblock flags in WMV7 typicallyreduces bitrate for many kinds of video content, it is less than optimalin some circumstances. In many cases, it fails to exploit redundancy inskip macroblock flags from macroblock to macroblock, for example, whenskipped macroblocks occur in bunches in a picture. Also, WMV7 ignoresmotion prediction for macroblocks in predicted frames when themacroblocks are skipped, which hurts the efficiency of compression ofpredicted frames in some cases.

[0029] C. Standards for Video Compression and Decompression

[0030] Aside from WMV7, several international standards relate to videocompression and decompression. These standards include the MotionPicture Experts Group [“MPEG”] 1, 2, and 4 standards and the H.261,H.262, and H.263 standards from the International TelecommunicationUnion [“ITU”]. Like WMV7, these standards use a combination ofintraframe and interframe compression, although the standards typicallydiffer from WMV7 in the details of the compression techniques used.

[0031] Some international standards recognize skipping coding ofmacroblocks as a tool to be used in video compression and decompression.For additional detail about skip macroblock coding in the standards, seethe standards' specifications themselves.

[0032] The skipped macroblock coding in the above standards typicallyreduces bitrate for many kinds of video content, but is less thanoptimal in some circumstances. In many cases, it fails to exploitredundancy in skip macroblock flags from macroblock to macroblock, forexample, when skipped macroblocks occur in bunches in a picture. Also,it ignores motion prediction for macroblocks in predictedmacroblocks/pictures when the macroblocks are skipped, which hurts theefficiency of compression of predicted macroblocks/pictures in somecases.

[0033] Given the critical importance of video compression anddecompression to digital video, it is not surprising that videocompression and decompression are richly developed fields. Whatever thebenefits of previous video compression and decompression techniques,however, they do not have the advantages of the following techniques andtools.

SUMMARY

[0034] In summary, the detailed description is directed to varioustechniques and tools for encoding and decoding (e.g., in a videoencoder/decoder) binary information. The binary information may comprisebits indicating whether a video encoder or decoder skips certainmacroblocks in a video frame. Or, the binary information may comprisebits indicating motion vector resolution for macroblocks (e.g. 1-MV or4-MV), interlace mode (e.g., field or frame), or some other information.Binary information may be encoded on a frame-by-frame basis or on someother basis.

[0035] In some embodiments, the binary information is arranged in a bitplane. For example, the bit plane is coded at the picture/frame layer.Alternatively, the binary information is arranged in some other wayand/or coded at a different layer. The encoder and decoder process thebinary information. The binary information may comprise macroblock-levelinformation. Alternatively, the encoder and decoder process bit planesof block-level, sub-block-level, or pixel-level information.

[0036] In some embodiments, the encoder and decoder switch coding modes.For example, the encoder and decoder use normal, row-skip, orcolumn-skip mode. The different modes allow the encoder and decoder toexploit redundancy in the binary information. Alternatively, the encoderand decoder use other and/or additional modes such as differentialmodes. To increase efficiency, the encoder and decoder may use a bitplane inversion technique in some modes.

[0037] In some embodiments, the encoder and decoder define a skippedmacroblock as a predicted macroblock whose motion is equal to itscausally predicted motion and which has zero residual error.Alternatively, the encoder and decoder define a skipped macroblock as apredicted macroblock with zero motion and zero residual error.

[0038] In some embodiments, the encoder and decoder use a raw codingmode to allow for low-latency applications. For example, in the rawcoding mode, encoded macroblocks can be transmitted to the decoder rightaway, without having to wait until all macroblocks in the frame/pictureare encoded. The encoder and decoder can switch between the raw codingmode and other modes.

[0039] The various techniques and tools can be used in combination orindependently. In particular, the application describes twoimplementations of skipped macroblock encoding and decoding, along withcorresponding bitstream syntaxes. Different embodiments implement one ormore of the described techniques and tools.

[0040] Additional features and advantages will be made apparent from thefollowing detailed description of different embodiments that proceedswith reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

[0041]FIG. 1 is a diagram showing block-based intraframe compression ofan 8×8 block of pixels according to prior art.

[0042]FIG. 2 is a diagram showing prediction of frequency coefficientsaccording to the prior art.

[0043]FIG. 3 is a diagram showing motion estimation in a video encoderaccording to the prior art.

[0044]FIG. 4 is a diagram showing block-based interframe compression foran 8×8 block of prediction residuals in a video encoder according to theprior art.

[0045]FIG. 5 is a diagram showing block-based intraframe decompressionfor an 8×8 block of prediction residuals according to the prior art.

[0046]FIG. 6 is a block diagram of a suitable computing environment inwhich several described embodiments may be implemented.

[0047]FIG. 7 is a block diagram of a generalized video encoder systemused in several described embodiments.

[0048]FIG. 8 is a block diagram of a generalized video decoder systemused in several described embodiments.

[0049]FIG. 9 is a chart showing the bitstream elements that make up theP picture layer according to the first implementation.

[0050]FIG. 10 is a flowchart showing a technique for encoding skippedmacroblock information in a video encoder having plural skip-macroblockcoding modes.

[0051]FIG. 11 is a flowchart showing a technique for decoding skippedmacroblock information encoded by a video encoder having pluralskip-macroblock coding modes.

[0052]FIG. 12 shows an example of a skipped macroblock coding frame.

[0053]FIG. 13 is a flowchart showing a technique for encoding in normalskip-macroblock coding mode.

[0054]FIG. 14 is a flowchart showing a technique for encoding in arow-prediction skip-macroblock coding mode.

[0055]FIG. 15 is a code listing showing pseudo-code for row-predictiondecoding of skipped macroblock information.

[0056]FIG. 16 is a flowchart showing a technique for encoding in acolumn-prediction skip-macroblock coding mode.

[0057]FIG. 17 is a code listing showing pseudo-code forcolumn-prediction decoding of skipped macroblock information.

[0058]FIG. 18 is a flowchart showing a technique for determining whetherto skip coding of certain macroblocks in a video encoder.

[0059]FIG. 19 is a flowchart showing a technique for encoding binaryinformation in a bit plane in a row-skip coding mode.

[0060]FIG. 20 is a flowchart showing a technique for encoding binaryinformation in a bit plane in a column-skip coding mode.

[0061]FIG. 21 is a flowchart showing a technique for encoding binaryinformation in a bit plane in a normal-2 coding mode.

[0062]FIGS. 22, 23 and 24 show examples of frames of binary informationtiled in normal-6 mode.

[0063]FIG. 25 is a flowchart showing a technique for encoding binaryinformation in a bit plane in a normal-6 coding mode.

[0064]FIG. 26 is a flowchart showing a technique for encoding binaryinformation in a differential coding mode.

[0065]FIG. 27 is a flowchart showing a technique for decoding binaryinformation encoded in a differential coding mode.

[0066]FIG. 28 is a flowchart showing a technique for selectivelyencoding binary information in raw coding mode for low latencyapplications.

DETAILED DESCRIPTION

[0067] Described embodiments relate to techniques and tools for encodingand decoding (e.g., in a video encoder/decoder) binary information. Thebinary information may comprise bits indicating whether a video encoderor decoder skips certain macroblocks in a video frame. Or, the binaryinformation may comprise bits indicating motion vector resolution formacroblocks (e.g. 1-MV or 4-MV), interlace mode (e.g., field or frame),or some other information. Binary information may be encoded on aframe-by-frame basis or on some other basis.

[0068] In some embodiments, the binary information is arranged in a bitplane. The bit plane is coded at the picture/frame layer. Alternatively,the binary information is arranged in some other way and/or coded at adifferent layer.

[0069] In some embodiments, the encoder and decoder switch coding modes.For example, the encoder and decoder use normal, row-skip, orcolumn-skip modes. The different modes allow the encoder and decoder toexploit redundancy in the binary information. Alternatively, the encoderand decoder use other and/or additional modes.

[0070] In some embodiments, the encoder and decoder define a skippedmacroblock as a predicted macroblock whose motion is equal to itscausally predicted motion and which has zero residual error.Alternatively, the encoder and decoder define a skipped macroblock as apredicted macroblock with zero motion and zero residual error.

[0071] In some embodiments, instead of efficient frame/picture-levelcoding, a raw coding mode is permitted to allow for low-latencyapplications. In the raw coding mode, encoded macroblocks can betransmitted to the decoder right away, without having to wait until allmacroblocks in the frame/picture are encoded.

[0072] In some embodiments, the encoder and decoder process bit planesof macroblock level information. Alternatively, the encoder and decoderprocess bit planes of block, sub-block, or pixel-level information.

[0073] The various techniques and tools can be used in combination orindependently. In particular, the application describes twoimplementations of skipped macroblock encoding and decoding, along withcorresponding bitstream syntaxes. Different embodiments implement one ormore of the described techniques and tools.

[0074] In described embodiments, the video encoder and decoder performvarious techniques. Although the operations for these techniques aretypically described in a particular, sequential order for the sake ofpresentation, it should be understood that this manner of descriptionencompasses minor rearrangements in the order of operations, unless aparticular ordering is required. For example, operations describedsequentially may in some cases be rearranged or performed concurrently.Moreover, for the sake of simplicity, flowcharts typically do not showthe various ways in which particular techniques can be used inconjunction with other techniques.

[0075] In described embodiments, the video encoder and decoder usevarious flags and signals in a bitstream. While specific flags andsignals are described, it should be understood that this manner ofdescription encompasses different conventions (e.g., 0's rather than1's) for the flags and signals.

[0076] I. Computing Environment

[0077]FIG. 6 illustrates a generalized example of a suitable computingenvironment (600) in which several of the described embodiments may beimplemented. The computing environment (600) is not intended to suggestany limitation as to scope of use or functionality, as the techniquesand tools may be implemented in diverse general-purpose orspecial-purpose computing environments.

[0078] With reference to FIG. 6, the computing environment (600)includes at least one processing unit (610) and memory (620). In FIG. 6,this most basic configuration (630) is included within a dashed line.The processing unit (610) executes computer-executable instructions andmay be a real or a virtual processor. In a multi-processing system,multiple processing units execute computer-executable instructions toincrease processing power. The memory (620) may be volatile memory(e.g., registers, cache, RAM), non-volatile memory (e.g., ROM, EEPROM,flash memory, etc.), or some combination of the two. The memory (620)stores software (680) implementing an encoder or decoder, such as avideo encoder or decoder.

[0079] A computing environment may have additional features. Forexample, the computing environment (600) includes storage (640), one ormore input devices (650), one or more output devices (660), and one ormore communication connections (670). An interconnection mechanism (notshown) such as a bus, controller, or network interconnects thecomponents of the computing environment (600). Typically, operatingsystem software (not shown) provides an operating environment for othersoftware executing in the computing environment (600), and coordinatesactivities of the components of the computing environment (600).

[0080] The storage (640) may be removable or non-removable, and includesmagnetic disks, magnetic tapes or cassettes, CD-ROMs, DVDs, or any othermedium which can be used to store information and which can be accessedwithin the computing environment (600). The storage (640) storesinstructions for the software (680) implementing the encoder or decoder.

[0081] The input device(s) (650) may be a touch input device such as akeyboard, mouse, pen, or trackball, a voice input device, a scanningdevice, or another device that provides input to the computingenvironment (600). For audio or video encoding, the input device(s)(650) may be a sound card, video card, TV tuner card, or similar devicethat accepts audio or video input in analog or digital form, or a CD-ROMor CD-RW that reads audio or video samples into the computingenvironment (600). The output device(s) (660) may be a display, printer,speaker, CD-writer, or another device that provides output from thecomputing environment (600).

[0082] The communication connection(s) (670) enable communication over acommunication medium to another computing entity. The communicationmedium conveys information such as computer-executable instructions,audio or video input or output, or other data in a modulated datasignal. A modulated data signal is a signal that has one or more of itscharacteristics set or changed in such a manner as to encode informationin the signal. By way of example, and not limitation, communicationmedia include wired or wireless techniques implemented with anelectrical, optical, RF, infrared, acoustic, or other carrier.

[0083] The techniques and tools can be described in the general contextof computer-readable media. Computer-readable media are any availablemedia that can be accessed within a computing environment. By way ofexample, and not limitation, with the computing environment (600),computer-readable media include memory (620), storage (640),communication media, and combinations of any of the above.

[0084] The techniques and tools can be described in the general contextof computer-executable instructions, such as those included in programmodules, being executed in a computing environment on a target real orvirtual processor. Generally, program modules include routines,programs, libraries, objects, classes, components, data structures, etc.that perform particular tasks or implement particular abstract datatypes. The functionality of the program modules may be combined or splitbetween program modules as desired in various embodiments.Computer-executable instructions for program modules may be executedwithin a local or distributed computing environment.

[0085] For the sake of presentation, the detailed description uses termslike “determine,” “select,” “reconstruct,” and “inform” to describecomputer operations in a computing environment. These terms arehigh-level abstractions for operations performed by a computer, andshould not be confused with acts performed by a human being. The actualcomputer operations corresponding to these terms vary depending onimplementation.

[0086] II. Generalized Video Encoder and Decoder

[0087]FIG. 7 is a block diagram of a generalized video encoder (700) andFIG. 8 is a block diagram of a generalized video decoder (800).

[0088] The relationships shown between modules within the encoder anddecoder indicate the main flow of information in the encoder anddecoder; other relationships are not shown for the sake of simplicity.In particular, FIGS. 7 and 8 usually do not show side informationindicating the encoder settings, modes, tables, etc. used for a videosequence, frame, macroblock, block, etc. Such side information is sentin the output bitstream, typically after entropy encoding of the sideinformation. The format of the output bitstream can be Windows MediaVideo version 8 format or another format.

[0089] The encoder (700) and decoder (800) are block-based and use a4:2:0 macroblock format with each macroblock including 4 luminance 8×8luminance blocks (at times treated as one 16×16 macroblock) and two 8×8chrominance blocks. Alternatively, the encoder (700) and decoder (800)are object-based, use a different macroblock or block format, or performoperations on sets of pixels of different size or configuration than 8×8blocks and 16×16 macroblocks.

[0090] Depending on implementation and the type of compression desired,modules of the encoder or decoder can be added, omitted, split intomultiple modules, combined with other modules, and/or replaced with likemodules. In alternative embodiments, encoder or decoders with differentmodules and/or other configurations of modules perform one or more ofthe described techniques.

[0091] A. Video Encoder

[0092]FIG. 7 is a block diagram of a general video encoder system (700).The encoder system (700) receives a sequence of video frames including acurrent frame (705), and produces compressed video information (795) asoutput. Particular embodiments of video encoders typically use avariation or supplemented version of the generalized encoder (700).

[0093] The encoder system (700) compresses predicted frames and keyframes. For the sake of presentation, FIG. 7 shows a path for key framesthrough the encoder system (700) and a path for forward-predictedframes. Many of the components of the encoder system (700) are used forcompressing both key frames and predicted frames. The exact operationsperformed by those components can vary depending on the type ofinformation being compressed.

[0094] A predicted frame [also called p-frame, b-frame forbi-directional prediction, or inter-coded frame] is represented in termsof prediction (or difference) from one or more other frames. Aprediction residual is the difference between what was predicted and theoriginal frame. In contrast, a key frame [also called i-frame,intra-coded frame] is compressed without reference to other frames.

[0095] If the current frame (705) is a forward-predicted frame, a motionestimator (710) estimates motion of macroblocks or other sets of pixelsof the current frame (705) with respect to a reference frame, which isthe reconstructed previous frame (725) buffered in the frame store(720). In alternative embodiments, the reference frame is a later frameor the current frame is bi-directionally predicted. The motion estimator(710) can estimate motion by pixel, ½ pixel, ¼ pixel, or otherincrements, and can switch the resolution of the motion estimation on aframe-by-frame basis or other basis. The resolution of the motionestimation can be the same or different horizontally and vertically. Themotion estimator (710) outputs as side information motion information(715) such as motion vectors. A motion compensator (730) applies themotion information (715) to the reconstructed previous frame (725) toform a motion-compensated current frame (735). The prediction is rarelyperfect, however, and the difference between the motion-compensatedcurrent frame (735) and the original current frame (705) is theprediction residual (745). Alternatively, a motion estimator and motioncompensator apply another type of motion estimation/compensation.

[0096] A frequency transformer (760) converts the spatial domain videoinformation into frequency domain (i.e., spectral) data. For block-basedvideo frames, the frequency transformer (760) applies a discrete cosinetransform [“DCT”] or variant of DCT to blocks of the pixel data orprediction residual data, producing blocks of DCT coefficients.Alternatively, the frequency transformer (760) applies anotherconventional frequency transform such as a Fourier transform or useswavelet or subband analysis. In embodiments in which the encoder usesspatial extrapolation (not shown in FIG. 7) to encode blocks of keyframes, the frequency transformer (760) can apply a re-orientedfrequency transform such as a skewed DCT to blocks of predictionresiduals for the key frame. In other embodiments, the frequencytransformer (760) applies an 8×8, 8×4, 4×8, or other size frequencytransforms (e.g., DCT) to prediction residuals for predicted frames.

[0097] A quantizer (770) then quantizes the blocks of spectral datacoefficients. The quantizer applies uniform, scalar quantization to thespectral data with a step-size that varies on a frame-by-frame basis orother basis. Alternatively, the quantizer applies another type ofquantization to the spectral data coefficients, for example, anon-uniform, vector, or non-adaptive quantization, or directly quantizesspatial domain data in an encoder system that does not use frequencytransformations. In addition to adaptive quantization, the encoder (700)can use frame dropping, adaptive filtering, or other techniques for ratecontrol.

[0098] If a given macroblock in a predicted frame has no information ofcertain types (e.g., no motion information for the macroblock and noresidual information), the encoder (700) may encode the macroblock as askipped macroblock. If so, the encoder signals the skipped macroblock inthe output bitstream of compressed video information (795).

[0099] When a reconstructed current frame is needed for subsequentmotion estimation/compensation, an inverse quantizer (776) performsinverse quantization on the quantized spectral data coefficients. Aninverse frequency transformer (766) then performs the inverse of theoperations of the frequency transformer (760), producing a reconstructedprediction residual (for a predicted frame) or a reconstructed keyframe. If the current frame (705) was a key frame, the reconstructed keyframe is taken as the reconstructed current frame (not shown). If thecurrent frame (705) was a predicted frame, the reconstructed predictionresidual is added to the motion-compensated current frame (735) to formthe reconstructed current frame. The frame store (720) buffers thereconstructed current frame for use in predicting the next frame. Insome embodiments, the encoder applies a deblocking filter to thereconstructed frame to adaptively smooth discontinuities in the blocksof the frame.

[0100] The entropy coder (780) compresses the output of the quantizer(770) as well as certain side information (e.g., motion information(715), spatial extrapolation modes, quantization step size). Typicalentropy coding techniques include arithmetic coding, differentialcoding, Huffman coding, run length coding, LZ coding, dictionary coding,and combinations of the above. The entropy coder (780) typically usesdifferent coding techniques for different kinds of information (e.g., DCcoefficients, AC coefficients, different kinds of side information), andcan choose from among multiple code tables within a particular codingtechnique.

[0101] The entropy coder (780) puts compressed video information (795)in the buffer (790). A buffer level indicator is fed back to bit rateadaptive modules.

[0102] The compressed video information (795) is depleted from thebuffer (790) at a constant or relatively constant bit rate and storedfor subsequent streaming at that bit rate. Therefore, the level of thebuffer (790) is primarily a function of the entropy of the filtered,quantized video information, which affects the efficiency of the entropycoding. Alternatively, the encoder system (700) streams compressed videoinformation immediately following compression, and the level of thebuffer (790) also depends on the rate at which information is depletedfrom the buffer (790) for transmission.

[0103] Before or after the buffer (790), the compressed videoinformation (795) can be channel coded for transmission over thenetwork. The channel coding can apply error detection and correctiondata to the compressed video information (795).

[0104] B. Video Decoder

[0105]FIG. 8 is a block diagram of a general video decoder system (800).The decoder system (800) receives information (895) for a compressedsequence of video frames and produces output including a reconstructedframe (805). Particular embodiments of video decoders typically use avariation or supplemented version of the generalized decoder (800).

[0106] The decoder system (800) decompresses predicted frames and keyframes. For the sake of presentation, FIG. 8 shows a path for key framesthrough the decoder system (800) and a path for forward-predictedframes. Many of the components of the decoder system (800) are used forcompressing both key frames and predicted frames. The exact operationsperformed by those components can vary depending on the type ofinformation being compressed.

[0107] A buffer (890) receives the information (895) for the compressedvideo sequence and makes the received information available to theentropy decoder (880). The buffer (890) typically receives theinformation at a rate that is fairly constant over time, and includes ajitter buffer to smooth short-term variations in bandwidth ortransmission. The buffer (890) can include a playback buffer and otherbuffers as well. Alternatively, the buffer (890) receives information ata varying rate. Before or after the buffer (890), the compressed videoinformation can be channel decoded and processed for error detection andcorrection.

[0108] The entropy decoder (880) entropy decodes entropy-coded quantizeddata as well as entropy-coded side information (e.g., motion information(815), spatial extrapolation modes, quantization step size), typicallyapplying the inverse of the entropy encoding performed in the encoder.Entropy decoding techniques include arithmetic decoding, differentialdecoding, Huffman decoding, run length decoding, LZ decoding, dictionarydecoding, and combinations of the above. The entropy decoder (880)frequently uses different decoding techniques for different kinds ofinformation (e.g., DC coefficients, AC coefficients, different kinds ofside information), and can choose from among multiple code tables withina particular decoding technique.

[0109] If the frame (805) to be reconstructed is a forward-predictedframe, a motion compensator (830) applies motion information (815) to areference frame (825) to form a prediction (835) of the frame (805)being reconstructed. For example, the motion compensator (830) uses amacroblock motion vector to find a macroblock in the reference frame(825). A frame buffer (820) stores previous reconstructed frames for useas reference frames. The motion compensator (830) can compensate formotion at pixel, ½ pixel, ¼ pixel, or other increments, and can switchthe resolution of the motion compensation on a frame-by-frame basis orother basis. The resolution of the motion compensation can be the sameor different horizontally and vertically. Alternatively, a motioncompensator applies another type of motion compensation. The predictionby the motion compensator is rarely perfect, so the decoder (800) alsoreconstructs prediction residuals.

[0110] When the decoder needs a reconstructed frame for subsequentmotion compensation, the frame store (820) buffers the reconstructedframe for use in predicting the next frame. In some embodiments, theencoder applies a deblocking filter to the reconstructed frame toadaptively smooth discontinuities in the blocks of the frame.

[0111] An inverse quantizer (870) inverse quantizes entropy-decodeddata. In general, the inverse quantizer applies uniform, scalar inversequantization to the entropy-decoded data with a step-size that varies ona frame-by-frame basis or other basis. Alternatively, the inversequantizer applies another type of inverse quantization to the data, forexample, a non-uniform, vector, or non-adaptive quantization, ordirectly inverse quantizes spatial domain data in a decoder system thatdoes not use inverse frequency transformations.

[0112] An inverse frequency transformer (860) converts the quantized,frequency domain data into spatial domain video information. Forblock-based video frames, the inverse frequency transformer (860)applies an inverse DCT [“IDCT”] or variant of IDCT to blocks of the DCTcoefficients, producing pixel data or prediction residual data for keyframes or predicted frames, respectively. Alternatively, the frequencytransformer (860) applies another conventional inverse frequencytransform such as a Fourier transform or uses wavelet or subbandsynthesis. In embodiments in which the decoder uses spatialextrapolation (not shown in FIG. 8) to decode blocks of key frames, theinverse frequency transformer (860) can apply a re-oriented inversefrequency transform such as a skewed IDCT to blocks of predictionresiduals for the key frame. In other embodiments, the inverse frequencytransformer (860) applies an 8×8, 8×4, 4×8, or other size inversefrequency transforms (e.g., IDCT) to prediction residuals for predictedframes.

[0113] When a skipped macroblock is signaled in the bitstream ofinformation (895) for a compressed sequence of video frames, the decoder(800) reconstructs the skipped macroblock without using the information(e.g., motion information and/or residual information) normally includedin the bitstream for non-skipped macroblocks.

[0114] III. First Implementation

[0115] In a first implementation, a video encoder and decoder encode anddecode, respectively, skipped macroblock information with improvedefficiency. The skipped macroblock information is signaled at thepicture layer in the video bitstream, which allows the encoder toexploit redundancy in the skipped macroblock information. Also, theencoder and decoder select between multiple coding modes for encodingand decoding the skipped macroblock information.

[0116] A. Picture Layer Coding of Skipped Macroblock Information

[0117] In the first implementation, a compressed video sequence is madeup of data structured into four hierarchical layers. From top to bottomthe layers are: 1) sequence layer; 2) picture layer; 3) macroblocklayer; and 4) block layer. At the picture layer, data for each pictureconsists of a picture header followed by data for the macroblock layer.(Similarly, at the macroblock layer, data for each macroblock consistsof a macroblock header followed by the block layer.) While some of thebitstream elements for I pictures and P pictures are identical, othersappear only in P pictures, and vice versa.

[0118]FIG. 9 shows the bitstream elements that make up the P-picturelayer (900). Table 1 briefly describes the bitstream elements of theP-picture layer (900). TABLE 1 Bitstream elements of the P-picture layerin first implementation Field Description PTYPE (910) Picture typePQUANT (912) Picture quantizer scale SMBC (920) Skipped macroblock codeSMB (930) Skipped macroblock field CPBTAB (940) Coded block patterntable MVRES (942) Motion vector resolution TTMBF (944) Macroblock-leveltransform type flag TTFRM (946) Frame-level transform type DCTACMBF(948) Macroblock-level DCT AC coding set flag DCTACFRM (950) Frame-levelDCT AC coding set index DCTDCTAB (952) Intra DCT DC table MVTAB (954)Motion vector table MB LAYER (960) Macroblock layer

[0119] In particular, the P-picture layer (900) includes a SkippedMacroblock field (“SMB”) (930) for the macroblocks in the P picture aswell as a Skipped Macroblock Code (“SMBC”) field (920) that signals thecoding mode for the skipped macroblock field (930). The SMBC field (920)is present only in P-picture headers. SMBC (920) is a 2-bit value thatsignals one of four modes used for indicating the skipped macroblocks inthe frame. In the first implementation, the fixed length codes (“FLCs”)for the skipped macroblock coding modes are as follows: TABLE 2 Skippedmacroblock coding mode code table in first implementation SMBC FLCSkipped Bit Coding Mode 00 No skipped bit coding 01 Normal skipped bitcoding 10 Row-prediction (or, “row-skip”) skipped bit coding 11Column-prediction (or, “column-skip”) skipped bit coding

[0120] If the coding mode is normal, row-prediction, orcolumn-prediction, then the next field in the bitstream is the SMB field(930) containing the skipped macroblock information. So, the SMB fieldis present only in P-picture headers and only if SMBC signals normal,row-prediction, or column-prediction skipped macroblock coding. If SMBCsignals normal coding, then the size of the SMB field is equal to thenumber of macroblocks in the frame. If SMBC signals row-prediction orcolumn-prediction, then the size of the SMB is variable as describedbelow.

[0121] The skipped macroblock information informs the decoder as towhich macroblocks in the frame are not present in the macroblock layer.For these macroblocks, the decoder will copy the correspondingmacroblock pixel data from the reference frame when reconstructing thatmacroblock.

[0122] B. Switching Coding Modes for Skipped Macroblock Information

[0123] As described above, the SMBC field (920) signals the coding modefor the skipped macroblock field (930). More generally, FIG. 10 shows atechnique (1000) for encoding skipped macroblock information in a videoencoder having multiple skip-macroblock coding modes. FIG. 11 shows acorresponding technique (1100) for decoding skipped macroblockinformation encoded by a video encoder having plural skip-macroblockcoding modes.

[0124] With reference to FIG. 10, the encoder selects a skip-macroblockcoding mode for coding skipped macroblock information (1010). Forexample, in the first implementation, the skipped macroblock codingmodes include a mode where no macroblocks are skipped, a normal mode, arow-prediction (or, “row-skip”) mode, and a column-prediction (or“column-skip”) mode. After the coding mode is selected, the encoderencodes the skipped macroblock information (1020). The encoder selectscoding modes on a picture-by-picture basis. Alternatively, the encoderselects coding modes on some other basis (e.g., at the sequence level).When the encoder is done encoding the skipped macroblock information(1030), encoding ends.

[0125] With reference to FIG. 11, the decoder determines theskip-macroblock coding mode used by the encoder to encode the skippedmacroblock information (1110). The decoder then decodes the skippedmacroblock information (1120). The decoder determines coding modes on apicture-by-picture basis. Alternatively, the decoder determines codingmodes on some other basis (e.g., at the sequence level). When thedecoder is done decoding the skipped macroblock information (1130),decoding ends.

[0126] C. Coding Modes

[0127] In the first implementation, the skipped macroblock coding modesinclude a mode where no macroblocks are skipped, a normal mode, arow-prediction (or, “row-skip”) mode, and a column-prediction (or“column-skip”) mode. The following sections describe how skippedmacroblock information is encoded in each mode with reference to FIG.12, which shows an example (1200) of a skipped macroblock coding frame.

1. Normal Skipped Macroblock Coding Mode

[0128] In normal mode, the skipped/not-skipped status of each macroblockis represented with a bit. Therefore, the size of the SMB field in bitsis equal to the number of macroblocks in the frame. The bit positionwithin the SMB field corresponds to the raster scan order of themacroblocks within the frame starting with the upper-left macroblock. Abit value of 0 indicates that the corresponding macroblock is notskipped; a bit value of 1 indicates that the corresponding macroblock isskipped.

[0129]FIG. 13 shows a technique (1300) for encoding in normalskip-macroblock coding mode. First, the encoder checks whether coding ofa macroblock will be skipped (1310). If so, the encoder adds a bit valueof 1 to the SMB field to indicate that the corresponding macroblock isskipped (1320). Otherwise, the encoder adds a bit value of 0 to the SMBfield to indicate that the corresponding macroblock is not skipped(1330). When the encoder is done adding bits to the SMB field (1340),skip macroblock coding ends.

[0130] As an example, using normal mode coding, the SMB field for theexample frame (1200) in FIG. 12 would be encoded as:010010111111111111010010.

2. Row-prediction Skipped Macroblock Coding Mode

[0131] In row-prediction mode, the status of each macroblock row (fromtop to bottom) is indicated with a bit. If the bit is 1, then the rowcontains all skipped macroblocks and the status for the next rowfollows. If the bit equals 0, then the skipped/not skipped status foreach macroblock in that row is signaled with a bit. Therefore, a bitfield equal in length to the number of macroblocks in a row follows. Thebits in the bit field represent the macroblocks in left-to-right order.Again, a value of 0 indicates that the corresponding macroblock is notskipped; a value of 1 indicates that the corresponding macroblock isskipped.

[0132]FIG. 14 shows a technique (1400) for encoding in row-prediction(or, “row-skip”) macroblock coding mode. First, the encoder checks if arow contains all skipped macroblocks (1410). If so, the encoder adds anindicator bit of 1 to the SMB field (1420) and the status for the nextrow follows. If the row does not contain all skipped macroblocks, theencoder adds an indicator bit of 0 to the SMB field, and theskipped/not-skipped status for each macroblock in that row is signaledwith a bit (1430). When the encoder is done with all the rows in theframe (1440), the row-prediction encoding ends.

[0133] As for decoding, FIG. 15 shows pseudo-code (1500) illustratingrow-prediction decoding of the skipped macroblock information. In thepseudo-code (1500), the function get_bits(n) reads n bits from thebitstream and returns the value.

[0134] As an example, using row-prediction mode coding, the SMB fieldfor the example frame (1200) in FIG. 12 would be encoded as:0010010110010010.

3. Column-prediction Skipped Macroblock Coding Mode

[0135] In column-prediction mode, the status of each macroblock column(from left to right) is indicated with a bit. If the bit is 1, then thecolumn contains all skipped macroblocks and the status for the nextcolumn follows. If the bit equals 0, then the skipped/not skipped statusfor each macroblock in that column is signaled with a bit. Therefore, abit field equal in length to the number of macroblocks in a columnfollows. The bits in the bit field represent the macroblocks intop-to-bottom order. Again, a value of 0 indicates that thecorresponding macroblock is not skipped; a value of 1 indicates that thecorresponding macroblock is skipped.

[0136]FIG. 16 shows a technique (1600) for encoding in column-prediction(or, “column-skip”) macroblock coding mode. First, the encoder checks ifthe column contains all skipped macroblocks (1610). If so, the encoderadds an indicator bit of 1 to the SMB field (1620) and the status forthe next column follows. If the column does not contain all skippedmacroblocks, the encoder adds an indicator bit of 0 to the SMB field,and the skipped/not-skipped status for each macroblock in that column issignaled with a bit (1630). When the encoder is done with all thecolumns in the frame (1640), the column-prediction encoding ends.

[0137] As for decoding, FIG. 17 shows pseudo-code (1700) illustratingcolumn-prediction decoding of the skipped macroblock information.

[0138] As an example, using column-prediction mode coding, the SMB fieldfor the example frame (1200) in FIG. 12 would be encoded as:0011010011000110100110.

[0139] IV. Second Implementation

[0140] In a second implementation, a video encoder and decoder encodeand decode, respectively, skipped macroblock information and/or other2-D binary data with improved efficiency. The encoder and decoder definea skipped macroblock as having a default motion (not necessarily zeromotion), which allows the encoder and decoder to skip more macroblocksin many cases. Efficient frame-level coding of bit planes indicatesskipped macroblock information and/or other 2-D binary data. Also, theencoder and decoder may use a raw (MB-level) coding option of skippedmacroblocks for low-latency applications.

[0141] A. Skip Bit Definition (Definition of Skipped Macroblock)

[0142] The second implementation includes a new definition of theconcept of a skipped macroblock. “Skip” refers to a condition in abitstream where no further information needs to be transmitted at thatlevel of granularity. A skipped macroblock (block) is a macroblock(block) that has a default type, default motion, and default residualerror. (In comparison, in other implementations and standards, skippedmacroblocks are predicted macroblocks with zero motion and zeroresiduals.

[0143] The new definition of skipped macroblock is a predictedmacroblock whose motion is equal to its causally predicted motion, andwhich has zero residual error. (The point of difference from the otherdefinition is the default motion is equal to the motion predictor, andthis may not necessarily be zero.)

[0144] For example, in some embodiments, predicted motion vectors for acurrent macroblock are taken from the macroblock directly above ordirectly to the left of the current macroblock. Or, horizontal andvertical components of the predictor are generated from the horizontaland vertical component-wise medians of the macroblocks the left, top,and top right of the current macroblock.

[0145] The motion vectors of a skipped macroblock with four motionvectors (4MV) are given by their predictions performed sequentially inthe natural scan order. As with the one motion vector (1MV) case, theerror residuals are zero.

[0146]FIG. 18 shows a technique (1800) for determining whether to skipcoding of particular macroblocks in a video encoder according to the newdefinition of skipped macroblocks. First, the encoder checks whether thecurrent frame is an I-frame or a P-frame (1810). If the current frame isan I-frame, no macroblocks in the current frame are skipped (1820), andskip-macroblock coding for the frame ends.

[0147] On the other hand, if the current frame is a P-frame, the encoderchecks for macroblocks in the current frame that can be skipped. For agiven macroblock, the encoder checks whether the motion vector for themacroblock is equal to the causally predicted motion vector for themacroblock (e.g., whether the differential motion vector for themacroblock is equal to zero) (1830). If the motion for a macroblock doesnot equal the causally predicted motion, the encoder does not skip themacroblock (1840). Otherwise, the encoder checks whether there is anyresidual to be encoded for the macroblock (1850). If there is a residualto be coded, the encoder does not skip the macroblock (1860). If thereis no residual for the macroblock, however, the encoder skips themacroblock (1870). The encoder continues to encode or skip macroblocksuntil encoding is done (1880).

[0148] B. Bit Plane Coding

[0149] In the second implementation, certain macroblock-specificinformation (including signaling skipped macroblocks) can be encoded inone bit per macroblock. The status for all macroblocks in a frame can becoded together as a bit plane and transmitted in the frame header.

[0150] In the second implementation, the encoder uses bit plane codingin three cases to signal information about macroblocks in a frame. Thethree cases are: 1) signaling skipped macroblocks, 2) signaling field orframe macroblock mode, and 3) signaling 1-MV or 4-MV motion vector modefor each macroblock. This section describes bit plane coding for any ofthe three cases and corresponding decoding.

[0151] Frame-level bit plane coding is used to encode two-dimensionalbinary arrays. The size of each array is rowMB×colMB, where rowMB andcolMB are the number of macroblock rows and columns, respectively.Within the bitstream, each array is coded as a set of consecutive bits.One of seven modes is used to encode each array, as enumerated in Table3 and described below. TABLE 3 Coding modes in second implementationCoding Mode Description Raw Coded as one bit per symbol Normal-2 Twosymbols coded jointly Diff-2 Differential coding of bit plane, followedby coding two residual symbols jointly Normal-6 Six symbols codedjointly Diff-6 Differential coding of bit plane, followed by coding sixresidual symbols jointly Row-skip One bit skip to signal rows with noset bits. Column-skip One bit skip to signal columns with no set bits.

[0152] In the second implementation, the encoder uses three syntaxelements to embed the information in a bit plane: MODE, INVERT andDATABITS.

[0153] The MODE field is a variable length code (“VLC”) that encodes thecoding mode for the bit plane. For example, the VLC in the MODE fieldrepresents any of the seven coding modes enumerated in Table 3. To savebits, the encoder can assign shorter codes to more probable coding modesand longer codes to less probable coding modes. As noted above, the MODEfield is transmitted in the frame header.

[0154] The encoder and decoder switch between coding modes on aframe-by-frame basis. For example, the encoder and decoder switchbetween coding modes as like the encoder and decoder of the firstimplementation switch between skipped macroblock coding modes in FIGS.10 and 11, respectively. Alternatively, the encoder and decoder switchusing some other technique and/or on some other basis.

[0155] If the mode is not raw mode, the one bit INVERT field is sent. Inseveral coding modes where conditional inversion may be performed, theINVERT field indicates whether the bits in the bit plane are to beinverted before encoding takes place in the encoder and whether theoutput of decoding in the decoder is to be inverted. The INVERT field is1 when most of the bits in the bit plane are equal to 1, and 0 when mostof the bits in the bit plane are equal to 0. The encoder employs severalcoding modes (such as normal-2 and normal-6) that consume less bits whenmore 0s are present. If the bit plane to be encoded has more 1s than 0s,the encoder can invert the bit plane to increase the proportion of 0s inthe bit plane and increase the potential for bit savings. Other modes(such as diff-2 and diff-6) use the value of the INVERT to calculate apredictor bit plane. Therefore, in some coding modes, the final bitplane reconstructed at the decoder depends on INVERT.

[0156] The DATABITS field is an entropy coded stream of VLC symbolscontaining the information necessary to reconstruct the bit plane, giventhe MODE and INVERT fields.

[0157] C. Coding Modes

[0158] In the second implementation, the encoder encodes binaryinformation (e.g., skipped macroblock information) in any of sevendifferent coding modes: row-skip mode, column-skip mode, normal-2 mode,normal-6 mode, diff-2 mode, diff-6 mode, and raw mode. A decoderperforms corresponding decoding for any of the seven coding modes. Eachmode is described in detail below.

[0159] Alternatively, the encoder and decoder use other and/oradditional coding modes.

1. Row-skip and Column-skip Modes

[0160] The row-skip coding mode saves bits by representing a row in abit plane with a single bit if each binary symbol in the row is of acertain value. For example, the encoder represents a skipped macroblockwith a 0 in a bit plane, and uses a row-skip coding mode that representsa row of all 0s with a single bit. The encoder therefore saves bits whenentire rows of macroblocks are skipped. The decoder performscorresponding decoding.

[0161] In the second implementation, all-zero rows are indicated usingone bit set to 0. When the row is not all zero, the one bit indicator isset to 1, and this is followed by colMB bits containing the bit planerow in order. Rows are scanned in the natural order.

[0162] Likewise, for the column-skip mode, if the entire row is zero, a0 bit is sent. Else, a 1 is sent, followed by the rowMB bits containingthe entire column, in order. Columns are scanned in the natural order.

[0163] For coding of leftover rows and/or columns in diff-6 and normal-6modes (described below), the same logic is applied. A one-bit flagindicates whether the row or column is all zero. If not, the entire rowor column is transmitted using one bit per symbol.

[0164] When the encoder encodes a bit plane consisting primarily of is,row-skip and column-skip coding are usually less efficient, because ofthe lower probability that rows/columns will consist entirely of 0s.However, the encoder can perform an inversion on the bit plane in such asituation to increase the proportion of 0s and potentially increase bitsavings. Thus, when conditional inversion is indicated through theINVERT bit, the encoder pre-inverts the bit plane before the bit planeis tiled and coded. On the decoder side, conditional inversion isimplemented by taking the inverse of the final output. (This is notperformed for the diff-2 and diff-6 mode.)

[0165]FIG. 19 shows a technique (1900) for encoding binary informationin a bit plane in a row-skip coding mode. The encoder first checkswhether inversion of the bit plane is appropriate, and, if so, performsthe inversion (1910). The encoder then checks a row in the bit plane tosee if each bit in the row is equal to 0 (1920). If so, the encoder setsthe indicator bit for the row to 0 (1930). If any of the bits in the roware not 0, the encoder sets the indicator bit for the row to 1 andencodes each bit in the row with one bit (1940). When the encoder isdone encoding all rows in the bit plane (1950), encoding of the bitplane ends.

[0166] A decoder performs corresponding decoding for the row-skip codingmode.

[0167]FIG. 20 shows a technique for encoding binary information in acolumn-skip coding mode. The encoder first checks whether inversion ofthe bit plane is appropriate, and, if so, performs the inversion (2010).The encoder then checks a column in the bit plane to see if each bit inthe column is equal to 0 (2020). If so, the encoder sets the indicatorbit for the column to 0 (2030). If any of the bits in the column are not0, the encoder sets the indicator bit for the column to 1 and encodeseach bit in the column with one bit (1940). When the encoder is doneencoding all columns in the bit plane (1950), encoding of the bit planeends.

[0168] A decoder performs corresponding decoding for the column-skipcoding mode.

2. Normal-2 Mode

[0169] The encoder uses the normal-2 mode to jointly encode pluralbinary symbols in a bit plane (e.g., by using a vector Huffman or othervariable length encoding scheme). The encoder encodes pairs of binarysymbols with variable length codes. The decoder performs correspondingdecoding.

[0170] If rowMB×colMB is odd, the first symbol is encoded as a singlebit. Subsequent symbols are encoded pairwise, in natural scan order. AVLC table is used to encode the symbol pairs to reduce overall entropy.

[0171] When conditional inversion is indicated through the INVERT bit,the encoder pre-inverts the bit plane before the bit plane is codedpairwise. On the decoder side, conditional inversion is implemented bytaking the inverse of the final output. (When the diff-2 mode is used,conditional inversion is not performed at this step.)

[0172]FIG. 21 shows a technique (2100) for encoding binary informationin normal-2 mode. The encoder performs an initial check to determinewhether inversion of the bit plane is appropriate to improve codingefficiency and, if so, performs the inversion (2110). The encoder thendetermines whether the bit plane being coded has an odd number of binarysymbols (2120). If so, the encoder encodes the first symbol with asingle bit (2130). The encoder then encodes symbol pairs with variablelength codes, using shorter codes to represent more probable pairs andlonger codes to represent less probable pairs (2140). When encoding ofthe symbol pairs is done (2150), the encoding ends.

[0173] A decoder performs corresponding decoding for the normal-2 codingmode.

3. Normal-6 Mode

[0174] The encoder also uses the normal-6 mode to jointly encode pluralbinary symbols in a bit plane (e.g., by using a vector Huffman or othervariable length encoding scheme). The encoder tiles groups of six binarysymbols and represents each group with a variable length code. Thedecoder performs corresponding decoding.

[0175] In the normal-6 mode (and the diff-6 mode), the bit plane isencoded in groups of six pixels. These pixels are grouped into either2×3 or 3×2 tiles. The bit plane is tiled maximally using a set of rules,and the remaining pixels are encoded using variants of row-skip andcolumn-skip modes.

[0176] In the second implementation, 3×2 “vertical” tiles are used ifand only if rowMB is a multiple of 3 and colMB is not a multiple of 3.Otherwise, 2×3 “horizontal” tiles are used. FIGS. 22, 23 and 24 showexamples of frames tiled in the normal-6 coding mode. FIG. 22 shows aframe (2200) with 3×2 vertical tiles and a 1-symbol wide remainder(shown as a shaded area) to be coded in column-skip mode. FIG. 23 showsa frame (2300) with 2×3 horizontal tiles and a 1-symbol wide remainderto be coded in row-skip mode. FIG. 24 shows a frame (2400) with 2×3horizontal tiles and 1-symbol wide remainders to be coded in row-skipand column-skip modes.

[0177] While 3×2 and 2×3 tiles are used in this example, in otherembodiments, different configurations of tiles and/or different tilingrules are used.

[0178] The 6-element tiles are encoded first, followed by thecolumn-skip and row-skip encoded linear tiles. If the array size is amultiple of 3×2 or 2×3, the latter linear tiles do not exist and the bitplane is perfectly tiled. The 6-element rectangular tiles are encodedusing a VLC table.

[0179] When conditional inversion is indicated through the INVERT bit,the encoder pre-inverts the bit plane before the bit plane is tiled andcoded. On the decoder side, conditional inversion is implemented bytaking the inverse of the final output. (When the diff-6 mode is used,conditional inversion is not performed at this step.)

[0180]FIG. 25 shows a technique (2500) for encoding binary informationin normal-6 mode. The encoder performs an initial check to determinewhether inversion of the bit plane is appropriate to improve codingefficiency and, if so, performs the inversion (2510). The encoder thenchecks whether the number of rows in the bit plane is a multiple ofthree (2520). If the number of rows is not a multiple of three, theencoder groups the symbols in the bit plane into 2×3 horizontal tiles(2530).

[0181] If the number of rows is a multiple of three, the encoder checkswhether the number of columns in the bit plane is a multiple of three(2540). If the number of columns is a multiple of three, the encodergroups the symbols in the bit plane into 2×3 horizontal tiles (2530). Ifthe number of columns is not a multiple of three, the encoder groups thesymbols into 3×2 vertical tiles (2550).

[0182] After grouping symbols into 3×2 or 2×3 tiles, the encoder encodesthe groups of six tiled symbols using a technique such as asix-dimension vector Huffman coding technique or some other codingtechnique (2560). The encoder encodes any remaining untiled symbolsusing the row-skip and/or column-skip coding techniques described above(2570).

[0183] A decoder performs corresponding decoding for the normal-6 codingmode.

[0184] In other embodiments, an encoder uses other techniques to codethe tiled and untiled symbols.

4. Diff-2 and Diff-6 Modes

[0185] Differential coding modes such as diff-2 or diff-6 mode encodebit planes by first generating a bit plane of differential (or residual)bits for the bit plane to be coded, based on a predictor for the bitplane to be coded. The residual bit plane is then encoded using, forexample, the normal-2 or normal-6 coding mode, without conditionalinversion.

[0186] In the second implementation, the diff-2 and diff-2 modes employdifferential coding denoted by the operation diff. If eitherdifferential mode is used, a bit plane of differential bits is firstgenerated by examining the predictor {circumflex over (b)}(i,j) of thebit plane b(i,j), which is defined as the causal operation:$\begin{matrix}{{\hat{b}\left( {,j} \right)} = \left\{ {\begin{matrix}{INVERT} \\{b\left( {0,{j - 1}} \right)} \\{b\left( {{i - 1},j} \right)}\end{matrix}\quad {\begin{matrix}{\quad {{ = {j = 0}},{{{or}\quad {b\left( {,{j - 1}} \right)}} \neq {b\left( {{ - 1},j} \right)}}}} \\{\quad { = 0}} \\{\quad {otherwise}}\end{matrix}.}} \right.} & (1)\end{matrix}$

[0187] In other words, the predictor {circumflex over (b)}(i,j) of agiven binary symbol b(i,j) will be the binary symbol just to the leftb(i−1,j) except in the following special cases:

[0188] 1) If b(i,j) is at the top left corner of the bit plane, or ifthe above binary symbol b(i,j−1) is not equal to the binary symbol tothe left b(i−1,j), the predictor {circumflex over (b)}(i,j) is equal tothe value of INVERT; or

[0189] 2) If 1) does not apply and b(i,j) is in the left column (i==0),the predictor {circumflex over (b)}(i,j) will be the above binary symbolb(i,j−1).

[0190] On the encoder side, the diff operation computes the residual bitplane r according to:

r(i,j)=b(i,j)⊕{circumflex over (b)}(i,j)  (2),

[0191] where ⊕ is the exclusive or operation. The residual bit plane isencoded using the normal-2 or normal-6 modes with no conditionalinversion.

[0192] On the decoder side, the residual bit plane is regenerated usingthe appropriate normal mode. Subsequently, the residual bits are used toregenerate the original bit plane as the binary 2-D difference:

b(i,j)=r(i,j)⊕{circumflex over (b)}(i,j)  (3).

[0193]FIG. 26 shows a technique (2600) for encoding binary informationin a differential coding mode. The encoder calculates a predictor for abit plane (2610), for example, as shown in equation 1. The encoder thencalculates a residual bit plane, for example, by performing an XORoperation on the bit plane and its predictor (2620). The encoder thenencodes the residual bit plane (e.g., in normal-2 or normal-6 mode)(2630).

[0194]FIG. 27 shows a technique (2700) for decoding binary informationencoded in a differential coding mode. The decoder decodes the residualbit plane (2710) using an appropriate decoding technique, based on themode used to encode the residual bit plane (e.g., normal-2 or normal-6mode). The decoder also calculates the predictor for the bit plane(2720), using the same technique used in the encoder. The decoder thenreconstructs the original bit plane, for example, by performing an XORoperation on the decoded residual bit plane and the predictor bit plane(2730).

5. Raw Mode

[0195] All modes except for raw mode encode a bit plane at the framelevel, which calls for a second pass through the frame during encoding.However, for low-latency situations, the second pass can addunacceptable delay (e.g., because transmission of the frame header andmacroblock layer information is delayed until the last macroblock in theframe is reached, because of the time spent encoding the bit plane).

[0196] The raw mode uses the traditional method of encoding the bitplane one bit per binary symbol at the same location in the bitstream asthe rest of the macroblock level information. Although macroblock-levelcoding of symbols is not a new concept by itself, switching the codingof symbols from frame level to macroblock level provides a low latencyalternative to frame-level coding.

[0197]FIG. 28 shows a technique (2800) for selectively encoding binaryinformation for a macroblock in raw coding mode for low latencyapplications. First, the encoder checks whether to use raw mode toencode the binary information (2810). If so, the encoder encodes a bitat the macroblock level for a macroblock (2820) and checks whether themacroblock is the last macroblock in the frame (2830). If the macroblockis not the last macroblock in the frame, the encoder continues byencoding a bit for the next macroblock at the macroblock level (2820).

[0198] If the encoder does not use the raw coding mode, the encoderencodes a bit plane at the frame level for the macroblocks in the frame(2840). When the encoding of the macroblocks in the frame is done(2850), the encoding ends for the frame.

[0199] While the technique (2800) shows switching modes on aframe-by-frame basis, alternatively the encoder switches on some otherbasis.

[0200] Having described and illustrated the principles of our inventionwith reference to various embodiments, it will be recognized that thevarious embodiments can be modified in arrangement and detail withoutdeparting from such principles. It should be understood that theprograms, processes, or methods described herein are not related orlimited to any particular type of computing environment, unlessindicated otherwise. Various types of general purpose or specializedcomputing environments may be used with or perform operations inaccordance with the teachings described herein. Elements of embodimentsshown in software may be implemented in hardware and vice versa.

[0201] In view of the many possible embodiments to which the principlesof our invention may be applied, we claim as our invention all suchembodiments as may come within the scope and spirit of the followingclaims and equivalents thereto.

We claim:
 1. In a computer system, a computer-implemented method ofprocessing one or more video images, the method comprising: selecting acoding mode from a group of plural available coding modes; andprocessing a bit plane according to the selected coding mode, whereinthe bit plane includes binary information for plural units of a videoimage, wherein the binary information represents characteristics of theplural units of the video image, wherein each of the plural unitsincludes plural pixels, and wherein the binary information includes onebit for each of the plural units of the video image.
 2. The method ofclaim 1 wherein the processing is performed at picture level.
 3. Themethod of claim 1 wherein the group of plural available coding modescomprises a row-prediction coding mode and a column-prediction codingmode.
 4. The method of claim 1 wherein the group of plural availablecoding modes further comprises one or more vector variable length codingmodes.
 5. The method of claim 1 wherein the group of plural availablecoding modes further comprises one or more differential coding modes. 6.The method of claim 1 wherein the plural units are macroblocks, andwherein the binary information in the bit plane comprises skippedmacroblock information.
 7. The method of claim 1 wherein the pluralunits are macroblocks, and wherein the binary information in the bitplane comprises motion vector count information.
 8. The method of claim1 wherein the plural units are macroblocks, and wherein the binaryinformation in the bit plane comprises field/frame flags.
 9. Acomputer-readable medium storing computer-executable instructions forcausing the computer system to perform the method of claim 1 duringvideo decoding, wherein the processing comprises entropy decodingaccording to the selected coding mode.
 10. A computer-readable mediumstoring computer-executable instructions for causing the computer systemto perform the method of claim 1 during video encoding, wherein theprocessing comprises entropy encoding according to the selected codingmode.
 11. In a computer system, a computer-implemented method ofprocessing plural video images in a video image sequence, the methodcomprising: selecting a coding mode from a group of plural availablecoding modes; and processing a bit plane according to the selectedcoding mode, wherein the bit plane includes binary informationsignifying whether macroblocks in a predicted video image are skipped ornot skipped, and wherein a macroblock in the predicted video image isskipped if the motion for the macroblock is equal to the predictedmotion of the macroblock and the macroblock has no residual error.
 12. Acomputer-readable medium storing computer-executable instructions forcausing the computer system to perform the method of claim 11 duringvideo encoding.
 13. In a computer system, a method of processing pluralvideo images of a video sequence, the plural video images including oneor more predicted video images, wherein each of the one or morepredicted video images includes plural macroblocks, the methodcomprising: processing one or more skipped macroblocks among the pluralmacroblocks of at least one of the one or more predicted video images,wherein each of the at least one of the one or more predicted videoimages is predicted from no more than one reference video image, whereineach of the one or more skipped macroblocks uses predicted motion forthe skipped macroblock based upon motion of one or more othermacroblocks, and wherein each of the one or more skipped macroblockslacks residual information.
 14. A computer-readable medium storingcomputer-executable instructions for causing the computer system toperform the method of claim 13 during video encoding.
 15. Acomputer-readable medium storing computer-executable instructions forcausing the computer system to perform the method of claim 13 duringvideo decoding.
 16. In a computer system, a method of processing pluralvideo images of a video sequence, the plural video images includingplural predicted macroblocks, wherein each of the plural predictedmacroblocks is predicted from no more than one reference video image,the method comprising: processing one or more skipped macroblocks amongthe plural predicted macroblocks, wherein each of the one or moreskipped macroblocks uses predicted motion for the skipped macroblockbased upon motion of one or more other predicted macroblocks, andwherein each of the one or more skipped macroblocks lacks residualinformation.
 17. A computer-readable medium storing computer-executableinstructions for causing the computer system to perform the method ofclaim 16 during video encoding.
 18. A computer-readable medium storingcomputer-executable instructions for causing the computer system toperform the method of claim 16 during video decoding.
 19. In a computersystem, a computer-implemented method of processing one or more imagesin a video image sequence, wherein a bitstream syntax for the videoimage sequence includes plural hierarchical layers, the pluralhierarchical layers including at least a sequence layer, a picturelayer, and a macroblock layer, the method comprising: processing a bitplane at a layer higher than the macroblock layer in the bitstreamsyntax for the video image sequence, wherein the bit plane includesbinary information for plural macroblocks in an image in the video imagesequence, wherein each of the plural macroblocks includes plural blocks,wherein each of the plural blocks includes plural pixels, and whereinthe binary information includes one bit for each of the pluralmacroblocks in the image.
 20. The method of claim 19 wherein the binaryinformation comprises skipped macroblock information.
 21. The method ofclaim 19 wherein the binary information comprises motion vector countinformation.
 22. The method of claim 19 wherein the binary informationcomprises interlace mode information.
 23. The method of claim 19 whereinthe higher layer is the picture layer.
 24. The method of claim 19wherein a header at the higher layer includes the bit plane duringtransmission.
 25. A computer-readable medium storing computer-executableinstructions for causing the computer system to perform the method ofclaim 19 during video encoding.
 26. A computer-readable medium storingcomputer-executable instructions for causing the computer system toperform the method of claim 19 during video decoding.
 27. In a computersystem, a computer-implemented method of processing one or more imagesin a video image sequence, wherein a bitstream syntax for the videoimage sequence includes plural hierarchical layers, the pluralhierarchical layers including at least a picture layer and a macroblocklayer, the method comprising: processing skipped macroblock informationat a layer higher than the macroblock layer in the bitstream syntax forthe video image sequence, wherein the skipped macroblock information isfor plural macroblocks in an image in the video image sequence, whereineach of the plural macroblocks includes plural blocks, wherein each ofthe plural blocks includes plural pixels.
 28. The method of claim 27wherein the higher layer is the picture layer.
 29. The method of claim27 wherein a header at the higher layer includes the skipped macroblockinformation.
 30. A computer-readable medium storing computer-executableinstructions for causing the computer system to perform the method ofclaim 27 during video encoding.
 31. A computer-readable medium storingcomputer-executable instructions for causing the computer system toperform the method of claim 27 during video decoding.
 32. In a computersystem, a computer-implemented method of processing plural images in avideo image sequence, wherein skipped macroblock information isassociated with the plural images, the method comprising: selecting acoding mode from a group of plural available coding modes, wherein atleast two of the plural available coding modes involve reduction ofbitrate associated with the skipped macroblock information; andprocessing the skipped macroblock information according to the selectedcoding mode.
 33. The method of claim 32 wherein the group of pluralavailable coding modes comprises a row-prediction mode, acolumn-prediction mode, and a normal mode.
 34. The method of claim 32wherein the group of plural available coding modes comprises a row-skipmode, a column-skip mode, a differential mode, an entropy coding mode,and a raw mode.
 35. The method of claim 32 wherein the processing is lowlatency processing, and wherein the selected coding mode is a one-passmacroblock-level coding mode.
 36. The method of claim 32 wherein abitstream syntax for the video image sequence includes pluralhierarchical layers, the plural hierarchical layers including a sequencelayer, a picture layer, and a macroblock layer, and wherein theprocessing occurs at the picture layer.
 37. A computer-readable mediumstoring computer-executable instructions for causing the computer systemto perform the method of claim 32 during video encoding.
 38. Acomputer-readable medium storing computer-executable instructions forcausing the computer system to perform the method of claim 32 duringvideo decoding.
 39. In a computer system, a computer-implemented methodof processing plural units in a video image sequence, wherein binaryinformation represents characteristics of plural units, wherein each ofthe plural units includes plural pixels, and wherein the binaryinformation includes one bit for each of the plural units, the methodcomprising: selecting a coding mode from a group of plural availablecoding modes, wherein the plural available coding modes include a lowlatency coding mode; and if the selected coding mode is the low latencycoding mode, processing the binary information according to the selectedcoding mode at a unit layer in a bitstream syntax for the video imagesequence, otherwise, processing the binary information as a bit planeaccording to the selected coding mode at a layer higher than the unitlayer in the bitstream syntax for the video image.
 40. Acomputer-readable medium storing computer-executable instructions forcausing the computer system to perform the method of claim 39 duringvideo encoding.
 41. A computer-readable medium storingcomputer-executable instructions for causing the computer system toperform the method of claim 39 during video decoding.
 42. In a computersystem, a computer-implemented method of processing plural units in avideo image sequence, wherein a bitstream syntax for the video imagesequence includes plural hierarchical layers, the plural hierarchicallayers including a sequence layer and a unit layer, the methodcomprising: selecting a coding mode from a group of plural availablecoding modes, wherein the plural available coding modes include a lowlatency coding mode; and if the selected coding mode is the low latencycoding mode, processing binary information according to the selectedcoding mode at the unit layer, otherwise, processing the binaryinformation as a bit plane according to the selected coding mode at alayer higher than the unit layer.
 43. A computer-readable medium storingcomputer-executable instructions for causing the computer system toperform the method of claim 42 during video encoding.
 44. Acomputer-readable medium storing computer-executable instructions forcausing the computer system to perform the method of claim 42 duringvideo decoding.
 45. In a computer system, a computer-implemented methodof compressing a bit plane comprising binary symbols in a matrixarranged in rows or columns, the method comprising: for each row in thebit plane where each binary symbol in the row is equal to a first value,or for each column in the bit plane where each binary symbol in thecolumn is equal to a first value, encoding the row or column with asingle bit in an encoded bit stream; and for each row in the bit planewhere at least one binary symbol in the row is not equal to the firstvalue, or for each column in the bit plane where at least one binarysymbol in the column is not equal to the first value, encoding the rowor column with one bit per binary symbol in the row or column.
 46. Themethod of claim 45 wherein the binary symbols in the matrix representskipped macroblock information in a video sequence.
 47. Acomputer-readable medium storing computer-executable instructions forcausing the computer system to perform the method of claim 45 duringvideo encoding.
 48. In a computer system, a computer-implemented methodof processing one or more video images, the method comprising:processing a bit plane according to a coding mode, wherein the bit planeincludes binary information for plural units of a video image, whereinthe binary information represents characteristics of the plural units ofthe video image, wherein each of the plural units includes pluralpixels, wherein the binary information includes one bit for each of theplural units of the video image, and wherein an invert flag indicateswhether to invert the bit plane as part of the processing.
 49. Acomputer-readable medium storing computer-executable instructions forcausing the computer system to perform the method of claim 48 duringvideo encoding, wherein the processing includes entropy encodingaccording to the coding mode, and wherein inversion of the bit planeoccurs, if at all, before the entropy encoding.
 50. A computer-readablemedium storing computer-executable instructions for causing the computersystem to perform the method of claim 48 during video decoding, whereinthe processing includes entropy decoding according to the coding mode,and wherein inversion of the bit plane occurs, if at all, after theentropy decoding.